{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction <a name=\"introduction\"></a>\n",
    "The notebook will **create a classification model using Scikit-learn, log facts using autologging,save model to Space using Watson Machine Learning**, **link model to model usecase**, **get and set environment types** for saved model asset.  \n",
    "<br/>\n",
    "For more information around model helper utilities see [here](https://ibm-aigov-facts-client.mybluemix.net/#model-helpers)\n",
    "\n",
    "It requires following services in the platform:\n",
    "\n",
    "- Watson Studio\n",
    "- Watson Machine Learning\n",
    "- Watson Knowledge catalog\n",
    "- AI factsheet\n",
    "- Facts client python sdk (>=1.0.42)\n",
    "\n",
    "Sections: \n",
    "\n",
    "- [Setup](#setup)\n",
    "    - [Cloud](#setupcloud)\n",
    "    - [Cloud Pak for data](#setupcpd)\n",
    "- [Initialize client](#init)\n",
    "    - [Cloud](#setupcloudinit)\n",
    "    - [Cloud Pak for data](#setupcpdinit)\n",
    "- [Create model using scikit learn](#createmodel)\n",
    "- [Save model in Watson machine learning space](#savemodel)\n",
    "- [Get model object](#getmodel)\n",
    "- [Add model to model usecase ](#add_mu)\n",
    "- [Get model current environment](#getenv)\n",
    "- [Set model environment ](#remove_mu)\n",
    "- [Cleanup](#clean)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name=\"setupcloud\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup <a href=\"#setup\"></a>\n",
    "### Imports\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -U ibm-aigov-facts-client --quiet\n",
    "!pip install -U ibm-watson-machine-learning --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "import shutil\n",
    "import time\n",
    "from ibm_watson_machine_learning import APIClient\n",
    "from ibm_aigov_facts_client import AIGovFactsClient\n",
    "from IPython.core.display import display, Markdown\n",
    "\n",
    "warnings.filterwarnings('ignore')\n",
    "shutil.rmtree('./mlruns', ignore_errors=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- This sample will use IBM Cloud by default. If you prefer to use Cloud pak for data, set `use_cp4d=True`\n",
    "- Flag `run_cleanup_at_end` offers option to delete created assets at the end of the notebook.The notebook will show URL to UI for model and model use case at certain cells. By dafault we set it to `run_cleanup_at_end=False` so you can access UI and see the changes. If you decide to cleanup assets at the end, set `run_cleanup_at_end=True` and remember cells showing links to UI will `NOT` work in that case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "use_cp4d=False\n",
    "run_cleanup_at_end=False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Model container type can be `SPACE` or `PROJECT`. However to use get/set environment utilities in this notebook, model asset need to be promoted to Space first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "EXPERIMENT_NAME=\"IrisClassification\"\n",
    "MODEL_NAME=\"IrisScikitModel\"\n",
    "CONTAINER_TYPE=\"space\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IBM Cloud <a href=\"#setupcloud\"></a>\n",
    "\n",
    "Your Cloud API key can be generated by going to the Users section of the Cloud console. From that page, go to **Manage->Access(IAM)->API keys-> Create**. Give your key a name and click Create, then copy the created key and use as API_KEY.\n",
    "\n",
    "NOTE: You can also get OpenScale API_KEY using IBM CLOUD CLI.\n",
    "\n",
    "How to install IBM Cloud (bluemix) console: instruction\n",
    "\n",
    "How to get api key using console:\n",
    "\n",
    "```\n",
    "bx login --sso\n",
    "bx iam api-key-create 'my_key'\n",
    "```\n",
    "- Get relevant space id from UI `(Deployments -> Spaces-> open space -> Manage -Space GUID)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not use_cp4d:\n",
    "    API_KEY=\"***\"\n",
    "    SPACE_ID=\"***\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[back to top](#introduction)\n",
    "## Cloud Pak for Data <a href=\"#setupcpd\"></a>\n",
    "- Service url is the Cloud pak for data platform host URL. For skytap environment, it would be the internal nginx URL.\n",
    "- You can either use user `password` or platform `apikey` to authenticate\n",
    "- Get relevant space id from UI `(Deployments -> Spaces-> open space -> Manage -Space GUID)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "if use_cp4d:\n",
    "    from ibm_aigov_facts_client import CloudPakforDataConfig\n",
    "\n",
    "    creds=CloudPakforDataConfig(service_url=\"***\",\n",
    "        username=\"***\",\n",
    "        api_key=\"***\")\n",
    "    \n",
    "    SPACE_ID=\"***\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[back to top](#introduction)\n",
    "# Client Initialization\n",
    "\n",
    "## IBM Cloud <a href=\"#setupcloudinit\"></a>\n",
    "\n",
    "- Container type would be either `space` or `project`. To use get/set environment utilities, model asset should be stored in Space.\n",
    "- If running this notebook multiple times with same experiment name or anytime face error saying `Experiment with same name already exists`, use `set_as_current_experiment=True` when initiating client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022/12/12 18:02:11 INFO : Experiment successfully created with ID 1 and name IrisClassification\n",
      "2022/12/12 18:02:12 INFO : Autolog enabled Successfully\n"
     ]
    }
   ],
   "source": [
    "if not use_cp4d:\n",
    "    facts_client = AIGovFactsClient(api_key=API_KEY, experiment_name= EXPERIMENT_NAME, container_type=CONTAINER_TYPE,container_id=SPACE_ID)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cloud Pak for Data <a href=\"#setupcpdinit\"></a>\n",
    "\n",
    "- Service url is the Cloud pak for data platform host URL. For skytap environment, it would be the internal nginx URL.\n",
    "- You can either use user password or platform apikey to authenticate\n",
    "- If running this notebook multiple times with same experiment name or anytime face error saying `Experiment with same name already exists`, use `set_as_current_experiment=True` when initiating client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "if use_cp4d:\n",
    "    facts_client = AIGovFactsClient(experiment_name= EXPERIMENT_NAME,container_type=CONTAINER_TYPE,container_id=SPACE_ID,cloud_pak_for_data_configs=creds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[back to top](#introduction)\n",
    "## Create model <a href=\"#createmodel\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022/12/12 18:02:14 INFO : logging results to factsheet for run_id ed6c56247dab41fda8ce45997e29e0e3\n",
      "2022/12/12 18:02:16 INFO : Successfully logged results to Factsheet service for run_id ed6c56247dab41fda8ce45997e29e0e3 under asset_id: 9bc27910-3cd7-4799-bbbd-8ec6ee6d268b and space_id : 2f3d2c21-9b7e-4d8f-af5d-c65276c71318\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from sklearn import svm, datasets\n",
    "import numpy as np\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "data1 = pd.DataFrame(data= np.c_[iris['data'], iris['target']],\n",
    "                    columns= ['sepal_length','sepal_width','petal_length','petal_width','target'])\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "train, test = train_test_split(data1, test_size = 0.4, random_state = 42)\n",
    "\n",
    "X_train = train[['sepal_length','sepal_width','petal_length','petal_width']]\n",
    "y_train = train.target\n",
    "X_test = test[['sepal_length','sepal_width','petal_length','petal_width']]\n",
    "y_test = test.target\n",
    "\n",
    "svc = svm.SVC(kernel=\"rbf\", C=2,probability=True)\n",
    "model=svc.fit(X_train,y_train)\n",
    "\n",
    "model.score(X_test, y_test)\n",
    "y_pred = model.predict(X_test)\n",
    "y_pred_prob = model.predict_proba(X_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[back to top](#introduction)\n",
    "# Store model to Space using Watson Machine Learning <a href=\"#savemodel\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "if use_cp4d:\n",
    "       WML_CREDENTIALS = {\n",
    "                   \"url\": creds.url,\n",
    "                   \"username\": creds.username,\n",
    "                   \"apikey\" : creds.api_key,\n",
    "                   \"instance_id\": \"wml_local\",\n",
    "                   \"version\" : \"4.5\"\n",
    "                  }\n",
    "else:\n",
    "       WML_CREDENTIALS = {\n",
    "                     \"url\": \"https://us-south.ml.cloud.ibm.com\",\n",
    "                     \"apikey\": API_KEY\n",
    "              }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0.253'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wml_client = APIClient(WML_CREDENTIALS)\n",
    "wml_client.version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'SUCCESS'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wml_client.set.default_space(SPACE_ID)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Software Specification ID: 12b83a17-24d8-5082-900f-0ab31fbfd3cb\n",
      "Storing model .....\n",
      "Done\n",
      "Model ID: 72d97d15-1e9c-4246-95ac-fbfdb519168c\n"
     ]
    }
   ],
   "source": [
    "software_spec_uid = wml_client.software_specifications.get_id_by_name(\"runtime-22.1-py3.9\")\n",
    "print(\"Software Specification ID: {}\".format(software_spec_uid))\n",
    "\n",
    "model_props = {\n",
    "    wml_client._models.ConfigurationMetaNames.NAME:\"{}\".format(MODEL_NAME),\n",
    "    wml_client._models.ConfigurationMetaNames.TYPE: \"scikit-learn_1.0\",\n",
    "    wml_client._models.ConfigurationMetaNames.SOFTWARE_SPEC_UID: software_spec_uid,\n",
    "    wml_client._models.ConfigurationMetaNames.LABEL_FIELD:\"target\",\n",
    "}\n",
    "\n",
    "facts_client.export_facts.prepare_model_meta(wml_client=wml_client,meta_props=model_props)\n",
    "\n",
    "print(\"Storing model .....\")\n",
    "\n",
    "published_model_details = wml_client.repository.store_model(model=model, meta_props=model_props, training_data=data1.drop([\"target\"], axis=1), training_target=data1.target)\n",
    "model_id = wml_client.repository.get_model_id(published_model_details)\n",
    "print(\"Done\")\n",
    "print(\"Model ID: {}\".format(model_id))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get model <a href=\"#getmodel\"></a>\n",
    "- It initates model object using Watson machine learning saved model details and returns a model object that allows you to use all helper methods [here](https://ibm-aigov-facts-client.mybluemix.net/#model-helpers)\n",
    "- `verbose=True` gives you additional informations about model asset\n",
    "- You can also use model_id to initate model object `facts_client.assets.get_model(model_id=<wml model id>)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022/12/12 18:02:23 INFO : Current model information: {'asset_id': '72d97d15-1e9c-4246-95ac-fbfdb519168c', 'container_type': 'space', 'container_id': '2f3d2c21-9b7e-4d8f-af5d-c65276c71318', 'facts_type': 'modelfacts_user'}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'name': 'IrisScikitModel',\n",
       " 'asset_type': 'wml_model',\n",
       " 'url': 'https://dataplatform.cloud.ibm.com/ml-runtime/models/72d97d15-1e9c-4246-95ac-fbfdb519168c?space_id=2f3d2c21-9b7e-4d8f-af5d-c65276c71318&context=cpdaas',\n",
       " 'asset_id': '72d97d15-1e9c-4246-95ac-fbfdb519168c',\n",
       " 'container_type': 'space',\n",
       " 'container_id': '2f3d2c21-9b7e-4d8f-af5d-c65276c71318',\n",
       " 'facts_type': 'modelfacts_user'}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wml_model=facts_client.assets.get_model(wml_stored_model_details=published_model_details)\n",
    "wml_model.get_info(verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[Click here to see the created wml model details in UI](https://dataplatform.cloud.ibm.com/ml-runtime/models/72d97d15-1e9c-4246-95ac-fbfdb519168c?space_id=2f3d2c21-9b7e-4d8f-af5d-c65276c71318&context=cpdaas)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_ui_url = wml_model.get_info(verbose=True)[\"url\"]\n",
    "display(Markdown(\"[Click here to see the created wml model details in UI](\" + model_ui_url + \")\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[back to top](#introduction)\n",
    "## Add model to new model usecase <a href=\"#add_mu\"></a>\n",
    "\n",
    "- Model usecase helps to track model asset lifecycle across different environments like development,pre-production and production.\n",
    "- You can link a model to existing model usecase by providing `model_usecase_id` or create a new usecase and link to that by providing `model_usecase_name` and `model_usecase_desc`.\n",
    "- You can link to existing model usecase by invoking `external_model.add_tracking_model_usecase(model_usecase_id=\"<model usecase id>\")`\n",
    "- If `model_usecase_catalog_id` is not given, it uses tha default available platform asset catalog id.\n",
    "- When you open the model usecase in model inventory, `model_usecase_id` refers to value after `asset/{ usecase id}` in browser URL.\n",
    "- You can also invoke `get_tracking_model_usecase()` to return currently tracked model usecase and get model usecase details by invoking `get_tracking_model_usecase().get_info()`\n",
    "- If using Cloud pak for data, make sure Openpages integration is disabled in your platform (create catalog permission needed). You can check it from UI `(Model inventory -> Manage)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022/12/12 18:02:24 INFO : Initiate linking model to new model use case......\n",
      "2022/12/12 18:02:38 INFO : Successfully finished linking Model 72d97d15-1e9c-4246-95ac-fbfdb519168c to model use case\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'model_entry_catalog_id': '745c21d7-ab6f-4408-9b3b-5d4760ccee1a',\n",
       " 'model_entry_id': 'aba647a6-f4ba-4133-bbe9-9e1c86f1263d',\n",
       " 'model_entry_name': 'ModelUsecaseDraft',\n",
       " 'model_entry_status': 'draft'}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wml_model.add_tracking_model_usecase(model_usecase_name=\"ModelUsecaseDraft\",model_usecase_desc=\"Draft model usecase for testing\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'model_usecase_id': 'aba647a6-f4ba-4133-bbe9-9e1c86f1263d',\n",
       " 'container_type': 'catalog',\n",
       " 'catalog_id': '745c21d7-ab6f-4408-9b3b-5d4760ccee1a',\n",
       " 'facts_type': 'model_entry_user'}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_usecase=wml_model.get_tracking_model_usecase()\n",
    "model_usecase.get_info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[back to top](#introduction)\n",
    "## Get model environment type <a href=\"#getenv\"></a>\n",
    "- When tracking is enabled, the recording of facts is triggered by actions on the model or related assets. Similarly, changes in the AI lifecycle control where the model display in a model usecase, to show the progression from development to operation. \n",
    "- For example, when a model is saved in a project, the model displays in the Develop pillar. When the model is promoted to a space, the entry is updated to show the model in the Test pillar, and so on.\n",
    "- `get_environment_type()` shows you where your asset exists and the reason behind it.\n",
    "- `TEST` environment is similar to `DEPLOY` in Cloud pak for data UI until `4.6.3` release. There is no difference in functionalities. From `4.6.3` onwards the environment will be called/showed as `TEST` in Cloud pak for data platform. \n",
    "\n",
    "\n",
    "Current expectations:\n",
    "\n",
    "| Model location | Develop | Test (show as Deploy in CPD <=4.6.2) | Validate | Operate |\n",
    "| --- | --- | --- |--- | --- |\n",
    "| Project | ✓ |\n",
    "| Space |  | ✓ |\n",
    "| Space tagged with AIGovernance: Pre-production | | | ✓ |\n",
    "| Space tagged with AIGovernance: Production | | | | ✓ |\n",
    "| Watson OpenScale Pre-production | | |✓ |\n",
    "| Watson OpenScale Production | | | | ✓ |\n",
    "\n",
    "- For external models, space concept is not applicable, however the method works as is considering your external model is saved as model stub in `Platform asset catalog` and linked to model usecase. You can follow steps as follows:\n",
    "    - Save mode stub using `save_external_model_asset()`. You can see example [here](https://ibm-aigov-facts-client.mybluemix.net/#externalmodelfactselements)\n",
    "    - Add to model usecase invoking `add_tracking_model_usecase()`\n",
    "    - Get current environment invoking `get_environment_type()`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'classification': 'TEST',\n",
       " 'reason': 'The space type is development which is considered as TEST environment and asset shows under TEST stage'}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wml_model.get_environment_type()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[back to top](#introduction)\n",
    "## Set model environment type <a href=\"#remove_mu\"></a>\n",
    "\n",
    "\n",
    "Following notes describe how the interaction between the lifecycle components controls where a Watson Machine Learning displays in a model entry.\n",
    "\n",
    "- When a model is promoted to a space, the model displays in the Test pillar. If the deployment for that model is evaluated in Watson OpenScale and tagged as pre-production, then the model displays in Validate rather than Test. If the model deployment is tagged as production instead of pre-production, then it moves to the Operate pillar.\n",
    "- If you assign the space tag `AIGovernance: Pre-production` or `AIGovernance: Production`, the model displays in the Validate or Operate pillar, respectively. Note: This change is triggered by an action resulting in a change to the model metadata, such as updating the model name, description, or tags, if the tag is added to the space after the model is tracked.\n",
    "- If a model is promoted to a space where the space has tag `AIGovernance: Pre-production`, the model displays in the Validate pillar. If the deployment for that model is evaluated in Watson OpenScale and tagged as pre-production, then the model displays in the Validate pillar. If the model deployment is tagged as production instead of pre-production, then it moves to the Operate pillar.\n",
    "- If a model is promoted to a space where the space has the tag AIGovernance: Production, it displays in the Operate pillar. If the deployment for this model is evaluated in Watson OpenScale and tagged as production, then it will be still in Operate pillar. But if it is tagged as Pre-production it displays in the Validate pillar.\n",
    "- If a model deployment is not evaluated in Watson OpenScale, it will have a Pending Evaluation tag. If the model deployment is evaluated, it will have an Evaluated tag. If the model deployment is approved after evaluation in Watson OpenScale, it will have an Approved tag.\n",
    "\n",
    "Notes:\n",
    "- If model is promoted to space and not monitored in Watson OpenScale, you can set model to specific environment pillar.\n",
    "- For external models, bahavior is identical."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022/12/12 18:02:46 INFO : Asset successfully moved from test to validate environment\n"
     ]
    }
   ],
   "source": [
    "time.sleep(3)\n",
    "wml_model.set_environment_type(from_container=\"test\",to_container=\"validate\")\n",
    "\n",
    "# uncomment if running cell by cell to see changes in UI\n",
    "\n",
    "# model_usecase_ui_url = model_usecase.get_info(verbose=True)[\"url\"]\n",
    "# display(Markdown(\"[Click here to see the model current environment in UI (Under Assets tab)](\" + model_usecase_ui_url + \")\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022/12/12 18:02:52 INFO : Asset successfully moved from validate to operate environment\n"
     ]
    }
   ],
   "source": [
    "time.sleep(3)\n",
    "wml_model.set_environment_type(from_container=\"validate\",to_container=\"operate\")\n",
    "\n",
    "# uncomment if running cell by cell to see changes in UI\n",
    "\n",
    "# model_usecase_ui_url = model_usecase.get_info(verbose=True)[\"url\"]\n",
    "# display(Markdown(\"[Click here to see the model current environment in UI (Under Assets tab)](\" + model_usecase_ui_url + \")\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022/12/12 18:02:59 INFO : Asset successfully moved from operate to validate environment\n"
     ]
    }
   ],
   "source": [
    "time.sleep(3)\n",
    "wml_model.set_environment_type(from_container=\"operate\",to_container=\"validate\")\n",
    "\n",
    "# uncomment if running cell by cell to see changes in UI\n",
    "\n",
    "# model_usecase_ui_url = model_usecase.get_info(verbose=True)[\"url\"]\n",
    "# display(Markdown(\"[Click here to see the model current environment in UI (Under Assets tab)](\" + model_usecase_ui_url + \")\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022/12/12 18:03:05 INFO : Asset successfully moved from validate to test environment\n"
     ]
    }
   ],
   "source": [
    "time.sleep(3)\n",
    "wml_model.set_environment_type(from_container=\"validate\",to_container=\"test\")\n",
    "\n",
    "# uncomment if running cell by cell to see changes in UI\n",
    "\n",
    "# model_usecase_ui_url = model_usecase.get_info(verbose=True)[\"url\"]\n",
    "# display(Markdown(\"[Click here to see the model current environment in UI (Under Assets tab)](\" + model_usecase_ui_url + \")\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleanup <a href=\"#clean\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "[Click here to see the created wml model details in UI](https://dataplatform.cloud.ibm.com/ml-runtime/models/72d97d15-1e9c-4246-95ac-fbfdb519168c?space_id=2f3d2c21-9b7e-4d8f-af5d-c65276c71318&context=cpdaas)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "[Click here to see the created model use case in UI](https://dataplatform.cloud.ibm.com/data/catalogs/745c21d7-ab6f-4408-9b3b-5d4760ccee1a/asset/aba647a6-f4ba-4133-bbe9-9e1c86f1263d?context=cpdaas)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "if run_cleanup_at_end:\n",
    "    # remove model and model usecase\n",
    "    model_usecase=wml_model.get_tracking_model_usecase()\n",
    "    wml_model.remove_tracking_model_usecase()\n",
    "    facts_client.assets.remove_asset(asset_id=model_usecase.get_info()[\"model_usecase_id\"],container_type=model_usecase.get_info()[\"container_type\"],container_id=model_usecase.get_info()[\"catalog_id\"])\n",
    "    facts_client.assets.remove_asset(asset_id=wml_model.get_info()[\"asset_id\"],container_type=wml_model.get_info()[\"container_type\"],container_id=wml_model.get_info()[\"container_id\"])\n",
    "\n",
    "else:\n",
    "    \n",
    "    model_ui_url = wml_model.get_info(verbose=True)[\"url\"]\n",
    "    display(Markdown(\"[Click here to see the created wml model details in UI](\" + model_ui_url + \")\"))\n",
    "    model_usecase_ui_url = model_usecase.get_info(verbose=True)[\"url\"]\n",
    "    display(Markdown(\"[Click here to see the created model use case in UI](\" + model_usecase_ui_url + \")\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[back to top](#introduction)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congratulations. You have successfully finished the end to end sample around get/set environment pillars helpers in model usecase."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright © 2020, 2022 IBM. This notebook and its source code are released under the terms of the MIT License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "671fb7aa161d7cd648bbbcad6b003541cd9cf13e8157e186cbe0090b82566204"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
